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C/^ , Résumé. Dans le cadre de l'estimation non paramétrique d'une fonction multidimen- 

^ ' sionnelle nous nous intéressons à l'adaptation structurelle. Nous supposons que la fonction 

c^ ' à estimer possède la structure « single-index » dans laquelle ni fonction de lien ni vec- 

teur d'indice ne sont connus. Nous proposons une nouvelle procédure qui s'adapte simul- 
tanément à l'indice inconnu ainsi qu'à la régularité de la fonction de lien. Nous présentons 
une inégalité d'oracle « locale » (définie par la semi-norme ponctuelle ) pour la procédure 
proposée, qui est ensuite utilisée pour obtenir la borne supérieure du risque maximal sous 



lO ' une hypothèse de régularité sur la fonction de lien. D'après la borne inférieure obtenue 



pour le risque minimax l'estimateur construit est un estimateur adaptatif optimal sur l'en- 
semble de classes considérées. Pour la même procédure on établit également une inégalité 



(^ ! d'oracle « globale » (en norme Lr, r < oo) et étudie sa performance sur les classes de 

en I Nikol'skii. Cette étude montre que la méthode proposée peut être appliquée à l'estimation 

de fonctions ayant une régularité inhomogène. 

Mots-clés. Estimation adaptative, Borne inférieure. Vitesse minimax. Inégalité d'oracle. 
Modèle single-index. Adaptation structurelle. Régularité inhomogène. 



Abstract. In the framework of nonparametric multivariate function estimation we 
are interested in structural adaptation. We assume that the function to be estimated 
possesses the "single-index" structure where neither the link function nor the index vector 
is known. We propose a novel procédure that adapts simultaneously to the unknown 
index and smoothness of link function. For the proposed procédure, we présent a "local" 
oracle inequality (described by the pointwise seminorm), which is then used to obtain the 
upper bound on the maximal risk under regularity assumption on the link function. The 
lower bound on the minimax risk shows that the constructed estimator is optimally rate 
adaptive over the considered range of classes. For the same procédure we also establish 
a "global" oracle inequality (under the L,. norm, r < cxd) and study its performance 
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over the Nikol'skii classes. This study shows that the proposée! method can be applied to 
estimating functions of inhomogeneous sinoothness. 

Keywords. Adaptive estimation, Lower bounds, Minimax rate, Oracle inequality. Single- 
index model. Structural adaptation, Inhomogeneous smoothness. 

Model and set-up. We observe a path {Y^{t),t G V} satisfying the équation 

Y,{dt) = F{t)dt + eW{dt), t = (ti,...,t,)G[-l,l]^ (1) 

where VI^ is a Brownian sheet and e G (0, 1) . We consider d = 2 except the second 
assertion of Theorem [2] concerning a lower bound for function estimation at a point. 
Additionally, we assume that the function F has the single-index structure, i.e. there 
exist an unknown link function / : M — )■ M and an index vector 6* G S^ such that 

F(x) = fix'^e*). (2) 

We suppose that / G F^ = {(7 : M — > M | sup^gjg \g{u)\ < M} for some M > , however 
its knowledge is not required for the estimation procédure. Our aim is to estimate the 
entire function F on [—1/2,1/2]^ or its value F{x) from the observation {Ys{t),t G 
T>} without any prior knowledge of the nuisance parameters / and 6* . The quality of 
estimation is measured by TZ^^\F,F) = 'E,p\\F — -F||r , where || ■ H^ is the L^ norm on 
[-1/2, l/2]2, r Ci [l,oo),orbythe"pointwise"risk 7^g(F, F) = (E^|F(x) -F(a;)n^/^ 

Objectives. The goal of our study is at least threefold. First, we seek an estimation 
procédure F{x),x G [—1/2,1/2]^, for F which could be applicable to any function F 
satisfying ([2]). Moreover, we want to bound the risk of this estimator uniformly over the 
set ¥m X §^. More precisely, we establish for F{x) the local oracle inequality : 

ni%F,F)<Cr4%ix), V/gFm, y9*ES\ Vx G [-1/2, l/2]2. (3) 

Hère the quantity Ajq, is completely determined by the function /, vector 9* and noise 
level e, while Cr is a numerical constant independent of F and e. Next, we apply this 
resuit to minimax adaptive estimation over the scale of E[(/3,L), Hôlder classes of func- 
tions, see Définition [TJ In particular, we find the minimax rate over ]HI(/3, L) x S^ and 
prove that our estimator F achieves that rate, i.e. is optimally adaptive. This resuit is 
quite surprising because, if 6* is fixed, say 6* = (1,0)''', then it is well known that an 
optimally adaptive estimator does not exist, see [5|. 

Note also that local oracle inequality Q allows us to bound from above the "global" 
risk as well : TZ^^\F,F) < Cr\\Aj^l,\\r ■ The latter is a global oracle inequality. As local 
oracle inequality ([3]) is a powerful tool for deriving minimax adaptive results in pointwise 
estimation, so global oracle inequality can be used for constructing adaptive estimators 
of the entire function F . We will consider the collection of Nikol'skii classes Np(/3, L), see 
Définition [21 where /3, L > and 1 < p < cxd. It is important to emphasize that thèse 



classes allow estiinating functions of inhomogeneous smoothness, i.e. those which can be 
very regular on some parts of the observation domain and rather irregular on the others. 

The adaptation to the unknown parameters 9* and / can be formulated in ternis 
of sélection from a spécial family of kernel estimators in the spirit of the Lepski and 
Goldenschluger-Lepski sélection rules, see p, y, |3[ . However, the proposed hère procédure 
is quite différent from the aforementioned ones, and it allows us to solve the problem of 
minimax adaptive estimation under the L^. losses over a collection of Nikol'skii classes. 

In Section [T] we explain the proposed sélection rule and give the oracle inequalities. 
Section [2] is devoted to the application of thèse results to minimax adaptive estimation. 

1 Oracle approach 

Let /C : M — > M be a function (kernel) satisfying / /C = 1. With any /C, any z eM., h E 
(0, 1] and / G ¥m we associate Ajcj{h, z) = supg^f^ \ô^^ /^(N ~ A/^)ifi'^) ~^ fi^))^'^\ ' 
a monotonous approximation error of the kernel smoother ô'^ J }C{[u — z]/ô)f{u)du . In 
particular, if the function / is uniformly continuons, theiaAicj{h, z) ^ as /i — > . We 
will assume that the kernels are compactly supported symmetric Lipschitz functions. 

Oracle estimator. For any y eM. define Aicj{h,y) = sup^^Q{2a)^^ Jy^^ Aicj{h, z)dz , 
and Aj^j(/i, •) := m.ax lAjcjih, ■) , Aicj{h, ■)\ . Define the oracle bandwidth : 

for any y G M h*^j{y) = snp {h e [e\l] : Vh A*^j{h,y) < \\IC\\^e^\n{l/e)}. (4) 

In what foUows we assume that e < exp l — max[l, (2M||/C||i||/C||j^^)^] \. This assumption 
provides the well-defined h'^j and can be relaxed in several ways. 

The quantity similar to the defined above /ij^j first appeared in |7|] in the context 
of the estimating univariate functions possessing inhomogeneous smoothness. Some years 
later this approach has been developed in [^ and |3| for multivariate function estimation. 
In thèse papers, the interested reader can find a detailed discussion of the oracle approach. 

CL — 1/1 h~^f) \ 

„ „ ], àei{E(^Qh-)) = 

h^^, and consider the family of kernel estimators 

J" = [Fie,h){-) = àei{E(e,h)) j K(E(e,h){t - ■))Ye{àt), {9, h)eS'x [e^ . 
Hère K{u,v) = }C{u)IC{v), where /C obeys the above conditions. Note that 



F(g,h){-) -^F 



F(9,h) ( 



Àr(0,\\IC\\te'h^^). (5) 



The choice 9 = 9* and h = h* := h*f^Ax^9*) leads to the oracle (depending of F\ 
estimator -F(6)*,h*)(-)- The meaning of this estimator is explained by the foUowing resuit. 



Proposition 1. For any {f,6*) G ¥m x S^ , r > 1 and e as above we hâve 
7^(j(F(e.,,.),F) < Cr\\}C\\le^Hl/e)/hlj{x^e*),\/x G [-1/2, 1/2]^, 



with Cr 



e(i + |ç|)' 



l/r 



, Ç 



Ar(0,l). 

This resuit means that the "oracle" knows tlie value of the index 6* and the optimal, 
up to ln(l/e), trade-off h* between the approximation error determined by A^ /(^*5 ') 9.nd 
the stochastic error provided by the kernel estimator with the bandwidth h*, cf. (E]). That 
explains why the "oracle" chooses the "estimator" F[e*,h*)- Below we propose a "real" 
(based on the observation) estimator -F(-), which mimics the oracle. The construction of 
the estimator F{-) is based on the data-driven sélection from the family J-". 

Sélection rule. For any 0, z/ G §^ and any h G [e^, 1] define the matrices 

' ~ z/'e< 0. 



'{e,h){u,h) 



2h{l+\v^e\) 

_Jfi2+y2)_ 
2(i+k'e|) 



2h(l+\u^ 9\) 



{e,h){u,h) 



E[-e,h){u,h), 



2(l+|l/T0|) 

It is easy to check that (4/i)^^ < det(i?(5i^ft)(j^^/j)) < {2h)^^. The corresponding kernel 
estimator is defined by -F(e,/i)(i.,ft)(a;) = àet{E(^0^h)(^^^h)) j K{E^e^h){u,h){t-x))Y^{àt) . For any 



T] G (0, 1] let TH(r7) = C{r, }C)eJri ^ ln(l/Ê:) , the constant C{r, /C) is given in p], page 7. 
Set He = {hk = 2-^ A; = 0, 1, . . . } n [e^, 1] and define for any 9 eS^ a.nd he Ue 



R 



(e,h) 



X) 



sup < sup 



\e,ri)(u,ri) 



[x] - F, 



(1^,^) 



x 



- TH(r/) 



For any x introduce the random set V{x) = {{0,h) G §^ x He : Ri^e,h){,x) < 0}, and 
let h = max{/i : {0,h) G V{x)} if V{x) 7^ 0. Note that there exists î9 G S^ such that 
{^,h) G Vix), since the set lie is finite. Dénote 9 := {0 G §^ : {9, h) G Vix)} . If 
V{x) 7^ , put 9 = 9 such that 9 E Q ; otherwise 6* := (1, 0)^ . If è' is not unique, let us 
make any measurable choice. For instance, one can choose 6' G O with the smallest first 
coordinate. Put as a final estimator F{x) 



^{ê,h)(^)'^^6^6 



h = sup Ih G Us 



^(è,h)\^> 



^(è,'n)^^' 



< 



TH(r/), Vr/ < /i, r^ G Ue) 



Theorem 1. Local and global oracle inequalities . For any {f,9*) G ¥m x S^ , r > 



(e,hy 



imtoe'Hy^) 



\ hlf{x^9*) 



+ Cr,2{M,JC)nLeJln{l/e),^x e 



1 ll2 
2'2. 



7^1^)(%),F) < a,i(/c) 



\ 



imt^sHnil/e) 



h* 



+ Cr,2iM,IC)\\}Cr^EJ\nil/e 



The constants Cr,i(/C) and Cr,2{M,}C) are given in ^], page 11. 



2 Adaptive estimation 

In this section we apply the local oracle inequality given by the first assertion of 
Theorem [T] to the pointwise adaptive estimation over Hôlder classes. Next, we use the 
global oracle inequality for adaptation over Nikol'skii classes. 

For any a > , dénote by nia , the maximal integer strictly less than a , and assume 
that there exists b > such that / z^K{z)dz = 0, Vj = 1, . . . , rrii, . 

Pointwise adaptive estimation. 

Définition 1. Let /3 > and L > . A function (7 : M — ^ R belongs to the Hôlder class 
EI(/3,L) , if g is nip -times continuously differentiable, ||5'''™'''||oo ^ L, \/m < niji , and 

g^'^^^t + h)- g^^'^^t)] < Lh^-""^, Vt G M, /i > 0. 
The aim is to estimate F{x) assuming that F G F(6) := lJ^<blJL>o^2(/3, -^) , where 
¥,{13, L) = {f : M^ ^ M I F{z) = f{z^e), f G e(/3,L), 6 G S^-^} , rf > 2. 
Note that b can be an arbitrary number, but it must be chosen a priory. 
Tfieorem 2. Let b > be fixed and the assumptions on the kernels hold. Then, for any 
/3<b, L>0,xe [-1/2, l/2]2, with MP,L) = Li/(2/3+i) (ey'ln(l/£))^^^^^^^^^ we hâve 



sup n^F^- f)<\\IC\\1 a,i(/C)^,(/3, L) + Cr,2iL, /C) e^\n{l/e) 

Moreover, for any P,L > 0, d >2 and any e > small enough, 

inf sup nii(F,F]>xij,{(3,L), 

where infimum is over ail estimators. Hère x is a constant independent of e and L . 

The estimator F,-^^. is minimax adaptive with respect to {¥a{/3,L), P < b, L > 0} . It 
is surprising, since if the index is known, then F(/3, L) = ]HI(/3, L) , and the problem can be 
reduced to the estimation of / at a point in the univariate Gaussian white noise model. 
As it is shown in [5|] the optimally rate adaptive estimator over {IHI(/3, L), (3 < b, L > 0} 
does not exist. 

Adaptive estimation under the Lj. losses. 

Définition 2. Let /9>0, L>0, pG [l,C)o) . A function g : M. ^ M. belongs to the 
Nikol'skii class Np{/3,L) , if g is m 13 -times continuously differentiable, 

||^(")||p < L, Vm = 1 < m;3 and \\g^"'^\- + h) - g^'^^'\-)\\p < Lh^-'^f' ,\/h > 0. 

We assume Np(/3, L) = B.{P, L) if p = 00. 



Hère the target of estimation is the function F obeying the assumption F G Fp(b) 
Fp(6) := U/3<6UL>oF2,p(/3,i:), where 

F,,p(/3,L) = {f : M'^ ^ M I F{z) = f{z'^d), f G Np(/3,L), 6 G S^^-^} . 

Theorem 3. Let b > be fixed and the above assumptions on the kernels hold. Then 

sup 7^(^)fe^ f) < \\IC\\lLCr,i{IC)^e{f3,L,p)+Cr,2{L,IC)e^\nil/6 
FeF2,p(/3,L) V ^ ' ^ / L 

for any L > 0, p > 1, p~^ < /? < 6, and r > 1. Hère x is an absolute constant, and 



Ve{P,L,p) 



Li/(2/3+i) (eJh^ 



1/2-1/r 



L/3-VP+1/2 £Jln(l/e) 



2/3 
2^ + 1 




(2/3 + l)p > r; 


-- [ln(l/.)] 

/3-l/p + l/r 
\ /3-1/P+1/2 


", (2/3 + l)p = r; 
(2/3 + l)p < r. 



Note that F2,p(/3,L) D Np(/3,L). Indeed, the class Np{f3,L) can be viewed as the class of 
functions F satisfying F{-) = f{6^-) with 6 = (1,0)^. Then, the problem of estimating 
such (2-variate) functions can be reduced to the estimation of univariate functions ob- 
served in the one-dimensional GWN modeL Thus, the rate of convergence for the latter 
problem, cf. [l|, l2| and the références therein, is also the lower bound for the minimax risk 
defined on F2,p(/3,L). Therefore the proposed estimator Fg-^-, is optimally rate adaptive 
whenever (2/3 + l)p < r. In the case (2/3 + l)p > r, we loose only a logarithmic factor with 
respect to the optimal rate, and the construction of optimally rate adaptive estimator 
over a collection |F2,p(/3,L), /3 > 0, L > 0\ in this case remains an open problem. 
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